Method of coding image segments of any shape

ABSTRACT

A method of coding digital image data of an original image for transmission and reproduction. The original image is subdivided into partial image regions whose shapes are adapted to local image structures. For each partial image region, a set of two-dimensional, lineraly independent basis functions is provided. The areal expanse of the basis functions approximately corresponds to the size of a rectangle circumscribing the partial image region to be coded and the raster of the sampled values of the basis functions corresponds to the pixel raster of the original image. The sampled values of the basis functions disposed within the area defined by the partial image region are orthogonalized in order to obtain a set of new, orthogonal basis functions, with the set including as many orthogonal basis functions as there are pixels within the partial image region. Coefficients of the orthogonalized basis functions describing the partial image region are then calculated. The coefficients correspond to the rectangular image regions in the partial image region and represent the gray scale values of the partial image region and thus an approximation of the original image. In a preferred embodiment, the contour of the partial image region is coded and transmitted along with the coefficients to a receiving location, where the orthogonal basis functions are generated based on the contour and the image is reproduced with the orthogonal basis functions and the coefficients.

REFERENCE TO RELATED APPLICATION

This application claims the priority of Federal Republic of GermanyApplication Serial No. P 39 33 346.9 filed Oct. 6th, 1989, which isincorporated herein by reference.

BACKGROUND OF THE INVENTION

The invention relates to a method of coding digital image data,particularly for the purpose of transmission over channels having alimited transmission capacity, while employing transform coding whichconverts the video image or partial regions thereof by means of atransformation rule into another representation. The representation ofthe image data is effected by a number of coefficients corresponding tothe number of pixels in the original image region. Such coefficients,however, have only a considerably lower correlation in contrast to thepixels of the video image which may have large correlations. That is,the coefficients are substantially uncorrelated while the pixels may begreatly correlated as in regions of the image where the gray scale valueis substantially uniform. The independence of the coefficients allowsfor a selection of a subquantity of coefficients, which may beconsiderably smaller than the number of original pixels. The selectioncan be based on the coefficient amplitudes, e.g. all coefficients whoseamplitudes exceed a predetermined magnitude. The selected coefficientsare quantized, which reduces the amount of data further. Using theselected and quantized coefficients produces, after reversal of thetransformation rule, an approximate reconstruction of the original imageregion.

It is known to process digital image data for transmission over channelshaving a limited transmission capacity by subjecting them to transformcoding, e.g. a DCT (discrete cosine transform) or a Walsh-Hadamardtransform as is described, for example in U.S. Pat. No. 4,805,017; andEsprit '86, Results and Achievements, Commission of the EuropeanCommunities, Directorate General XIII, Telecommunications, Information,Industries & Innovation, Elsevier Science Publishers B.V., 1987 (NorthHolland), pages 413-422. All prior art methods of transform coding havein common that the images or partial image regions (segments) to besubjected to transformation have a rectangular or often even squareshape. The images are divided into blocks by a regular grid and theseblocks are transform coded separately. The condition that onlyrectangular partial image regions can be transformed has the resultthat, on the one hand, interference patterns in the form of thesubdivision predetermined by the block grid, so-called blocking effects,occur and, on the other hand, uniform regions are unnecessarilysubdivided and thus the attainable data compression is limited.

According to Digital Image Processing, William K. Pratt, published byJohn Wiley & Sons, New York, N.Y., U.S.A., pages 232-278, the imageproduced by the transformation can be represented by the weighted sum ofa set of basis functions. The basis functions are here fixed for theentire image (see, for example, page 245, FIG. 10.3-2, Cosine BasisFunctions) and are given by the type of transformation. The basisfunctions may, for example be in the form of polynomials ortrigonometric functions.

SUMMARY OF THE INVENTION

It is an object of the invention to provide an improved digital datacoding method of the above described type according to which no annoyingblocking effects occur and data compression is not limited by a divisioninto blocks. This is accomplished by transform coding partial imageregions of any selected shape whose contours may be, for example,adapted to local image structures.

For each partial image region, a set of two-dimensional, linearlyindependent functions, so-called basis functions, equal in number to atmost the number of pixels in the partial image region, is provided.These basis functions represent, for example, the sampled values oftwo-dimensional polynomials or trigonometric functions. The arealexpanse of the basis functions approximately correspond to the size of arectangle circumscribing the segment or partial region to be coded, andthe raster of the sampled values of the basis functions corresponds tothe pixel raster of the original image. The sampled values of the basisfunctions disposed within the area defined by the partial image regionare orthogonalized in order to obtain a set of new, orthogonal basisfunctions, with the set including at most as many basis functions asthere are pixels within the partial image region. Coefficients of theorthogonalized basis functions describing the partial image region arethen calculated. The coefficients correspond to the weights of therespective basis functions in the partial image region, which representthe gray scale values of the partial image region and thus anapproximation of the original partial image region.

In a preferred embodiment, the contour of the partial image region iscoded using suitable techniques as described in the literature, andalong with the coefficients is transmitted to a receiving location wherethe orthogonal basis functions are generated based on the contour andthe image reproduced with the orthogonal basis functions and thecoefficients.

The invention is based in part on the realization that in contrast toprevious transformation methods, the partial image regions (segments)which are subjected to the transformation rule may have any desiredshape and may, in particular, be adapted to the local image content insuch a manner that the local image content of each respective segment islimited to similar structures. Due to these similarities, imagestructures of the individual segment can be defined by a smaller numberof coefficients than in the prior art methods. Moreover, thesecoefficients may be quantized in a rough manner for further datareduction. An additional means for allowing for data reduction isobtained by using a smaller number of coefficients than there are pixelsin the segment as, for example, by using a lesser number of originalbasis functions in obtaining the orthogonal base functions and/orignoring some of the coefficients obtained using the orthogonal basisfunctions, and it becomes unnecessary to process redundant image datafor uniform image regions. Moreover, the shapes of the segments are nottied to any conditions whatsoever, even multiply connected regions(segments containing holes) being permissible.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the invention will be furtherunderstood from the following detailed description of the preferredembodiments with reference to the accompanying drawings in which:

FIG. 1 shows DCT basis images;

FIG. 2 gives an example of a two-value window function including a gridwhich symbolizes the basic sampling raster;

FIG. 3 shows basis images of nonorthogonal starter polynomials;

FIG. 4 shows basis images of orthogonal polynomials;

FIG. 5 is an illustration of an original segment;

FIG. 6 is an illustration of a reconstructed segment;

FIG. 7 is a block diagram of a coder according to the invention;

FIG. 8 is a block diagram of a decoder according to the invention; and

FIG. 9 is a flow chart for a progressive reconstructive build-up of animage according to the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Before describing the invention in detail, the basic principle oftransform coding will first be discussed. Transform coding involves arule for imaging from the domain of the pixels into a coefficientdomain, also called the spectral range. The transformation as such hereresults in no data reduction and is rather an unequivocalrepresentation. That means that the original image can be reconstructedwithout errors from the coefficient representation. Rather, theadvantage of transformation lies in the reduction of correlationsexisting between the pixels in the original image. Moreover, withincreasing order in the original image, the amplitudes of thecoefficients decrease quickly from their highest levels so that anapproximate description of the original image may be effected with asub-quantity of coefficients in that, for example, only the coefficientshaving the greatest amplitudes are transmitted to a receiver. This ofcourse results in the ability to represent correlations between thepixels with reduced data and therefore reduced bandwidth. The receiverperforms the inverse transformation, with non-transmitted coefficientsbeing assumed to be zero. The real value coefficients can additionallybe quantized, again introducing an error. Thus, depending on the numberof transmitted coefficients and on the quantization of the amplitudes, amore or less falsified reconstruction of the original image is obtainedin the receiver. However, at the same time, a much smaller data rate isrequired for transmission than would have been the case for faithfullyreproducing the original image.

The image or image function represented by the transformation can beillustrated very clearly with the aid of basis images (two-dimensionalbasis functions). The image function is represented by a weighted sum ofbasis images. The basis images are here predetermined according to thetype of transformation to be employed. FIG. 1 shows a set of basisimages as defined by the discrete cosine transform (DCT) for squareimage regions (blocks) of a size of 8 pixels by 8 pixels. The weightingfactors for the basis images are the coefficients. By appropriateselection of coefficients, any desired gray scale value distribution inthe block can be represented. Since the finer structures occur lessfrequently and are looked upon with much more tolerance by the humaneye, it is usually sufficient to form a weighted combination of lowfrequency basis images as they are shown in the upper left portion ofFIG. 1.

Mathematically, the representation of a reconstructed image blockg(x_(k),y_(l)) can be expressed as follows by N-basis functionsφn(x_(k),y_(l)), n=1 to N:

    g(x.sub.k,y.sub.l)=a.sub.1 φ.sub.1 (x.sub.k,y.sub.l)+a.sub.2 φ.sub.2 (x.sub.k,y.sub.l)+ . . . +a.sub.N φ.sub.N(x.sub.k,y.sub.l)

The basis functions φ₁, φ₂, . . . φ_(N) are determined, as mentionedabove, by the transformation. The desired parameter vector ofcoefficients a^(N) =[a₁, a₂, . . . , a_(N) ] minimizes the square of theerror between the reconstruction or reconstructed image function g andthe original image function f: ##EQU1##

To accomplish this, partial differentiations are made in a known mannerwith respect to the components a_(q) of parameter vector a^(N) and thederivatives are set to zero: ##EQU2##

These equations are called normal equations. In order to determine thecoefficients, that is the parameter vector a^(N) =[a₁, a₂, . . . a_(N)], it is necessary to solve the normal equations. The solvability of theequation system composed of N equations is determined by the number M ofthe given pixels within an image block and the number N of the unknowncoefficients in the parameter vector a^(N). The following cases can bedistinguished:

1. M<N: The parameters cannot be determined unequivocally. The equationsystem is therefore under-determined.

2. M=N: For the case of linearly independent basis functions, preciselyone solution of the equation system can be guaranteed. The given datavalues are represented in a basis function system, the matrix of thebasis functions Φ. The determination of parameters a involves atransformation into another coordinate system. By employing the cosinefunction as the basis function system, the known DCT (discrete cosinetransform) is obtained.

3. M>N: In this case, the original image function f(x_(k),y_(l)) and thereconstruction g(x_(k),y_(l)) will coincide only in exceptional casessince the equation system has more equations than unknowns. Graphicallyspeaking, an attempt is made to have a line defined by two parameterspass through more than two points. The equation system isover-determined and can be solved by distributing the error to allsupporting locations in the sense of minimizing the mean total error(least mean square=LMS).

Over-determined systems permit a compact description with respect todata compression since the given function values are described by asmaller number of parameters than existing data values. Moreover, thistype of equalization of an error is able to reduce the effect ofstochastic errors in the data points.

The equations of the system of normal equations to be solved aremutually coupled and not necessarily mathematically well conditioned. Inorder to obtain a solution for the parameter vector a^(N), theGauss-Jordan algorithm (Bronstein/Semendjajew, Taschenbuch derMathematik ["Pocketbook of Mathematics"], published by Verlag HerriDeutsch, 23rd Edition, pages 735-737) may be employed. However, theGauss-Jordan algorithm is relatively complicated from a computationalpoint of view. The complexity results from the coupled nature of theequations which of course is not changed by the Gauss-Jordan algorithm.The coupling of the equations is expressed, for example, in the factthat, in an attempt to improve the approximation by adding furthercoefficients, all equations must again be solved. In other words, theaddition or omission of even one coefficient changes all othercoefficients. An improvement in an existing approximation may be ofsignificance for applications involving increased image build-up oradaptive methods.

The solution of the normalized equations is simplified mathematically toa considerably extent if a set of orthogonal basis functions is employedwhich meets the following condition for orthogonality: ##EQU3##

The resulting coefficients are then uncorrelated. Therefore, orthogonalbasis functions are employed exclusively. For example, the basisfunctions of the DCT shown in FIG. 1 are orthogonal.

Due to the orthogonality, the equation system is no longer coupled andthe normal equations ##EQU4## are simplified to ##EQU5##

The desired coefficients a_(q) can thus be calculated directly: ##EQU6##The orthogonal basis functions are orthonormal if the following applies:##EQU7## Thus, the calculation of the coefficients is then furthersimplified to ##EQU8##

Due to the introduction of orthogonality, the coefficients are obtainedby representing the image function in the basis function system φ_(q)(x_(k),y_(l)). In particular, if further coefficients are added, thevalues of all previously obtained coefficients now remain unchangedsince the calculations of the coefficients take place independently ofone another in a so-called hierarchic structure of the description.

The problem of determining orthogonal basis functions for any desiredshape of an image region will now be described. For this purposeorthogonalization is effected with respect to two-dimensional regions ofthe desired shape. Two functions φ_(n) and φ_(q) are orthogonal to oneanother if their scalar product disappears according to the aboveequation (1) for orthogonality. Orthogonalization is described, forexample, by R. Zurmuhl in Praktische Mathematik fur Ingenieure undPhysiker, ("Practical Mathematics for Engineers and Physicists"), 3rdEdition, 1961, pages 360-362. In order to restrict orthogonality of twodimensional functions to a specific region of the x,y plane, atwo-dimensional window function w(x,y) is introduced. This function hasvalues different from zero only within the specific region. An exampleof a two-value window function with respect to a region having anarbitrary boundary shape is shown in FIG. 2. The sampling raster (gridin FIG. 2) corresponds precisely to the pixel raster of the originalimage. With the aid of the window function the equation defining thecondition for orthogonality becomes: ##EQU9##

The summation forming the reconstructed image function g now producesamounts different from zero only within the respective image segment;accordingly, the functions φ_(n) and φ_(q) are orthogonal with respectto the shape of the segment, expressed by the window function. It shouldbe noted that the use of multi-value window functions permits theintroduction of local weighting. However, this is not employed herebelow.

It is clear at once that two given functions cannot simultaneously beorthogonal with respect to differently shaped regions, except for thecase of trivial solutions. On the other hand, image segments vary withrespect to the shape and number of pixels they enclose. Consequently, aspecial set of orthogonal basis functions must be found for eachindividual image segment. In this connection, an image block representsa special segment shape for which orthogonal basis functions need to bedetermined only once unless the size of the block is varied.

The following theorem guarantees that only one set of orthogonal basisfunctions exists for each segment shape.

For a linearly independent set of functions u₁, uu₂, . . . , uu_(N) inan N-dimensional sub-space A^(N), there exists a set of functions qq₁,q₂, . . . , q_(N), which are orthogonal with respect to the samesub-space A^(N). These orthogonal functions q₁, q₂, . . . , q_(N) caneven be represented as linear combinations of the given functions

    u.sub.1, u.sub.2, . . . , u.sub.N.

The proof for this theorem can be found in the literature. For anydesired segment shape it is possible to find a linearly independent setof starting functions. The computational derivation of the orthogonalbasis functions from the linearly independent, non-orthogonal startingfunctions can be effected with the aid of the known mathematical methodsof Schmidt (N. I. Achiser, Vorlesungen uber Approximations-theorie("Lectures on Approximation Theory"), 2nd Edition, 1967, published byAkademie Verlag, pages 24-26) or Householder (A. S. Householder,Principles of Numerical Analysis, McGraw-Hill Book Company, 1953, pages72-78). Both methods furnish the same orthogonal functions except fordifferent constant multiplication factors. In contrast to theHouseholder method, the Schmidt method permits the derivation of singleindividual functions which are orthogonal to the already calculatedfunctions. The example of an image segment will serve as an explanationfor an orthogonalization employing the Schmidt algorithm. If one employsthe simple, linearly independent polynomials 1, x, y, x², x·y, y² asstarting functions and bases the procedure on a rectangular samplingraster, the orthogonalization method furnishes the known Gram-Schmidtpolynomials. These have a great similarity in their shape to Legendrepolynomials which are orthogonal only with respect to the continuouscase. The use of trigonometric functions thus constitutes a special casein that the functions are orthogonal with respect to rectangular imagesegments in the discrete case as well as in the continuous case. Forarbitrarily shaped image segments, however, a method oforthogonalization such as have been described must be employed even ifsine or cosine functions are employed.

FIG. 3 shows basis images for the non-orthogonal basis functions x^(i)·y^(j) for a given segment shape. The basis functions are defined on thesmallest square surrounding the segment; the value range of thecoordinates covers the interval [-1,+1] in the vertical as well as inthe horizontal direction. The degree of the illustrated basis functionsascends from zero (constant) in the upper left-hand corner, for x in thehorizontal direction and for y in the vertical direction. Next to theconstant basis function, one therefore sees the respective functions φ₁₀=x (horizontal ramp) and φ₀₁ =y (vertical ramp). The values of theremaining basis functions for higher powers of x and y exhibit littlevariation within the segment since the greatest changes occur in thedirection of the edges, that is for |x|, |y|→1.

The basis functions illustrated here make immediately apparent theexistence of substantial difficulties in the use of non-orthogonal basisfunctions of a higher degree than "one". Thus, the applicable literatureoften unjustifiably considers the polynomial approximation to be animage coding method that is not very promising, or it is reported thatthe use of higher degrees of polynomials does not bring any furtherimprovement. The consequence is then often a segmentation into anexcessively large number of small segments which can be describedsufficiently accurately by means of polynomials of no higher degree thanthe first order. However, this procedure is in contradiction to theexpected advantage of coding larger segments. At the same time, thequantity of data required for coding the shape or contour of the regionsincreases considerably.

Basis images corresponding to the polynomials obtained afterorthogonalization are illustrated in FIG. 4. The structures representedby the various basis images are clearly discernible. The figure makesapparent the suitability of these orthogonal functions for imagedescription. The set of new basis functions includes at most as manyfunctions as there are pixels within an image segment. For manyapplications, the number of basis functions in the original set, and theequal number of orthogonal basis functions in the new set, aresubstantially smaller than the number of pixels. After the orthogonalbasis functions have been obtained, they are used in transform codingthe image region (segment) in the same manner as for rectangulartransforms according to equation (2) above, where the summation is onlycarried out over the region of the segment, as described by the windowfunction.

FIG. 5 shows the gray scale values within the original image segment.The reconstructions obtained by means of orthogonal polynomials of atmost the ninth degree is shown in FIG. 6. The figure clearly shows thenoise reducing effect of the approximation. At the same time, structureswithin the segment are reproduced far better than would have beenpossible, for example, with polynomials of up to the second degree.

A set of orthogonal basis functions can be orthogonal only with respectto a special shape of a region and not simultaneously with respect toseveral shapes. However, that generally means that it is necessary totransmit not only a code representing the contour to describe therespective boundaries of the image segments and the coefficients forapproximating the gray scale values in the interior but also the basisfunction system being employed in order to enable the receiver toreconstruct the gray scale values. The "overhead" created by thetransmission of the basis function system, however, would most likelycompensate for the gain in region oriented coding. A coder structurewhich does not require transmission of the basis function system willnow be described.

FIG. 7 is a block diagram of a coder structure which may be used toperform the method according to the invention. Based on a givensegmentation, the contour (boundaries) of each segment and the grayscale values within each segment are separately coded.

The coder for the contours serves the purpose of representing theboundaries of each segment as efficiently as possible. Generally, onedistinguishes between exact coding, for example, run length coding, andapproximative coding with respect to a selected degree of error. Therelatively high bit requirement is a drawback of exact coding; theliterature speaks of approximately 1.4 bits per contour point. In thecase of approximative coding, additional algorithms are required toeliminate the occurrence of overlaps between adjacent regions or"no-man's-land" between regions. Moreover, the contour must bereconstructed in the transmitter so that the orthogonalization processcan be performed on the contour obtained in the receiver. Known methodsmay be employed to obtain the contour code and the correspondingreconstruction at the receiving end, for example, the method by E. L.Hall, described in Computer Image Processing and Recognition, AcademicPress, pages 413-420 (7.3.1 Boundary Description) or the methoddescribed by E. K. Jain in Fundamentals of Digital Image Processing,Prentice Hall, pages 362-374 (9.6. Boundary Representation).

The location of the partial regions inside the complete image can becoded by employing a reference point for each partial region. An exampleof such a reference point is the center of gravity of the partialregion. Another example is the topmost point belonging to that regionor, if there is more than one topmost point, the leftmost of these. Thelocation (e.g. coordinate inside image frame) of the reference point iscoded and sent along with the data describing the contour and thecontent of the partial region to the receiver.

The coding of gray scale values within the region is effected segment bysegment. For example, coding of the segment marked A in FIG. 7 takesplace as follows: With the aid of a set of linearly independent basisfunctions Pij, the so-called basic knowledge, a set of orthogonal basisfunctions P_(i) ^(A) j is generated with reference to the shape ofsegment A. To do this, the orthogonalization process requires thecontour information and may be performed using the window function w, asdescribed above, according to the procedure described in thepublications of Schmidt or Householder referred to above. The thusobtained orthogonal basis functions are utilized to define the grayscale values within the segment A. Segment A is completely described bythe transmission of the contour code and the approximation coefficients.

The block circuit diagram of the decoder is shown in FIG. 8. Initially,the shape of segment A is reconstructed from the transmitted contourinformation. Since the receiver is equipped with the same basicknowledge (the set of linearly independent basis functions P_(i) j), theorthogonal basis functions can be generated analogously to the procedureat the transmitter based on the shape of the segment.

Once the shape of the segment has been reconstructed, the orthogonalityequations are set up with the aid of the window function as describedabove. The orthogonal basis functions are generated by means of theabove mentioned methods of Schmidt or Householder. Characteristic of theuse of the window function and thus of the orthogonalization withrespect to a two-dimensional shape is that in the calculation of theorthogonal basis functions only sampled values of the linearlyindependent, non-orthogonal basis functions which lie within theboundaries (contour) of the segment are employed.

The additional transmission of the orthogonal basis functions for eachsegment is redundant since the transmitted contour code contains theinformation on how to construct the basis functions. Finally, the grayscale values within the segment are reconstructed by a weighted sum ofthe orthogonal basis functions decoded from the transmitted contour codeand approximation coefficients. The approximation coefficients hereconstitute the weighting factors used to provide an approximatereconstruction of the gray scale values by the decoder at the receiver.

In summary, the coding and decoding of each individual segment takesplace as follows:

Coding:

1. description of the contour of the segment being observed to obtainthe contour code;

2. generation of a set of orthogonal basis functions for the specificshape;

3. approximation of the gray scale values in the interior of the segmentwith approximation coefficients.

Transmission:

1. transmission of the contour code;

2. transmission of the approximation coefficients.

Decoding:

1. reconstruction of the shape of the segment;

2. generation of a set of orthogonal basis functions for the specificshape of the segment;

3. reconstruction of the gray scale values within the segment.

Different parameters of this coding scheme may be adapted to the actualuse. Several parameters relating to the coding of the gray scale valueswill now be described in greater detail.

The set of linearly independent but non-orthogonal starting functions,called the "basic knowledge", is of course not limited to polynomialssuch as are used in the embodiment of FIGS. 7 and 8. It can bedemonstrated that the only condition placed on the starting functions istheir linear independence which, however, does not constitute asignificant restriction. Thus, it is possible to use any of a multitudeof different sets of basis functions as the starting point for theorthogonalization. This provides the advantage that the functionsemployed can be individually adapted to each respective segment. Thus,segments with soft luminance transitions can be advantageouslyrepresented with polynomials while Walsh functions are better suited forcoding regions containing, for example, text. The use of cosinefunctions would correspond to use of the DCT and could furnish goodresults for segments containing periodic variations in gray scalevalues.

In order for the receiver to be informed which of a predetermined numberof types of basis functions is being used for a particular partialregion, a code, which references the type of basis functions to be usedfor that particular partial region, is sent to the receiver in additionto the other data being sent.

A further parameter for adapting the method to actual use is the degreeof quantization of the individual coefficients. The quantization of thecoefficients is equivalent to the case of a block-oriented transform.Many different techniques are described in the literature, for example,Section 4.4.3 of the JPEG (Joint Photographic Experts Group) draftstandard, which has been submitted to ISO (JPEG--8-R4, ISO/IECJTCl/SC2/WG8 CCITT SGVIII, Aug. 31, 1989). It can be especiallyadvantageous, to consider an adaption of the employed quantization tothe order of the coefficient and therefore to the basis function,represented by the coefficient. It has already been stated that thehuman visual system is less susceptible to higher spatial frequencies.Therefore, coefficients corresponding to basis functions showing higherspatial frequencies are quantized more coarsely than others. Bothtransmitter and receiver agree on the type of quantizer to be used foreach coefficient.

Still another parameter for adapting the method to actual use, andrelated to quantization, is the number of approximation coefficientsemployed per segment or, its equivalent, the accuracy of thereconstruction. Due to the use of orthogonal basis functions, if certaincoefficients have been transmitted to provide a particular order ofapproximation to the original image (resolution), and it is decided toimprove the approximation, additional coefficients may be transmittedand the earlier transmitted coefficients remain unchanged. This not onlycorresponds to the principle of an hierarchical representation but alsooffers the opportunity to be discussed below of a "growing" imagebuild-up, interaction with the user and adaptation to the content of theimage.

The technology of progressive image build-up is employed, for example,in the transmission of image data over low bit rate channels. Initiallyan observer is shown a rough (coarse) image or a lowpass filtered imagewhich is reconstructed at the end of a first image build-up phase.Detail information is added successively during further image build-upphases--by determining additional basis functions of a higher order andcorresponding additional coefficients--until the desired resolution hasbeen realized. In contrast to a conventional sequential line-by-linebuild-up of the image, with a growing image build-up according to theinvention, the observer receives useful information right at thebeginning of the transmission. Due to the orthogonality of thedetermined basis functions, the coefficients determined during previousimage build-up phases retain their validity and need not be computedanew. One example in this connection is the call-up of images from adata bank. Often, the user can decide in an early stage of imagebuild-up whether the image contains the desired information. The user isthen able, in interaction with the system at each stage, to causefurther detail information to be transmitted or stop the transmissionand thus save time and money.

FIG. 9 is a flow chart illustrating the method in which the quality ofthe reconstruction is improved in steps. The treatment of the contourand the derivation of the orthogonal basis functions is identical tothat effected by the coder structure shown in FIG. 7. However, twofurther steps are also required. Reconstruction of the gray scale valuesin the transmitter permits a comparison of the reconstruction with thegray scale values of the original segment. If the quality of thereconstruction is insufficient, an improved approximation can beobtained adaptively, or by user interaction by transmitting higher orderbasis functions to the receiver, as described above.

If a hybrid codec (coder-decoder) is employed to code video imagesequences by forming residual error images, each of which is formed by adifference between the momentary image of the video sequence and anestimated image generated from the preceding coded and decoded image,only those partial image regions of each residual error image are codedwhich are distinguished by particularly high error amplitudes. Suchpartial image regions may have any shape. This use of the hybrid codecavoids the previously customary description of the residual error imageby means of block-shaped partial image regions which resulted because ofthe exclusive use of a block oriented transformation rule. Thus, thisuse prevents the unnecessary transmission of pixels with negligiblysmall error amplitudes within image blocks containing residual errorpixels with nonnegligible error amplitudes.

For the case that only a sub-quantity of coefficients is to betransmitted to the receiver, as described above, the receiver is to beinformed which coefficients have been selected. One possibility is thetransmission of the coefficients in an ordered way, e.g. in a scanningfashion (refer to FIG. 1), and sending a small code for each coefficientthat has been set to zero, that is, has been omitted. Another solutionis described in Section 4.4.4 of the above-mentioned JPEG (JointPhotographic Experts Group) document. It will be understood that theabove description of the present invention is susceptible to variousmodifications, changes and adaptations, and the same are intended to becomprehended within the meaning and range of equivalents of the appendedclaims.

What is claimed is:
 1. A method of coding digital image data of anoriginal video image, comprising the steps of:(1) subdividing theoriginal image into partial image regions of respective shapes whosecontours differ; (2) for each partial image region, determining sampledvalues of a set of initial two-dimensional, linearly independent basisfunctions of the image data in the partial image region, the basisfunctions being defined over an area including at least the whole areaof the partial image region, the sampled values of the basis functionshaving a raster corresponding to the pixel raster of the original image;(3) for each partial image region, orthogonalizing the sampled values ofthe basis functions disposed within the area defined by the partialimage region in order to obtain a new set of orthogonal basis functions,with the new set including, at most, as many orthogonal basis functionsas there are pixels within the partial image region; and (4) for eachpartial image region, determining and coding respective coefficients ofthe new set of orthogonal basis functions, a sum of the orthogonal basisfunctions multiplied by the respective coefficients representing thegray scale values of the partial image region, the coded coefficientsrepresenting an approximation of the partial image region; and,furthercomprising, in an initial build-up phase and at least one furtherbuild-up phase, the steps of transmitting the coded digital image datato a receiving location and reconstructing the transmitted image data atthe receiving location, wherein for each partial image region a videoimage sequence is coded and transmitted in such a manner that at thereceiving location a first reconstructed image is determined at the endof the initial image build-up phase from a first number of theorthogonal basis functions and a first number of the coefficients, thefirst number being substantially less than the number of pixels in thepartial image region, and, at the end of a second image build-up phase,an improved reconstruction of the image is obtained from a second numberof the orthogonal basis functions and a second number of thecoefficients, the second number of the orthogonal basis functions beingof a higher order than the order of the first number of orthogonal basisfunctions.
 2. A method as defined in claim 1, wherein the number ofimage build-up phases is controlled as a function of a desired imageresolution.
 3. A method of coding digital image data of an originalvideo image which is defined over an area of arbitrary shape, comprisingthe steps of:determining sampled values of a set of initialtwo-dimensional, linearly independent basis functions of the image datain the original image, the basis functions being defined over an area atleast including the whole area of the original image, the sampled valuesof the basis functions having a raster corresponding to the pixel rasterof the original image; orthogonalizing the sampled values of the basisfunctions disposed within the area defined by the original image inorder to obtain a new set of orthogonal basis functions, with the newset including, at most, as many orthogonal basis functions as there arepixels within the original image; and determining and coding respectivecoefficients of the new set of orthogonal basis functions, a sum of theorthogonal basis functions multiplied by the respective coefficientsrepresenting the gray scale values of the original image, the codedcoefficients representing an approximation of the original image.
 4. Amethod of coding digital image data of an original video image,comprising the steps of:(1) subdividing the original image into partialimage regions of respective shapes whose contours differ; (2) for eachpartial image region, determining sampled values of a set of initialtwo-dimensional, linearly independent basis functions of the image datain the partial image region, the basis functions being defined over anarea including at least the whole area of the partial image region, thesampled values of the basis functions having a raster corresponding tothe pixel raster of the original image; (3) for each partial imageregion, orthogonalizing the sampled values of the basis functionsdisposed within the area defined by the partial image region in order toobtain a new set of orthogonal basis functions, with the new setincluding, at most, as many orthogonal basis functions as there arepixels within the partial image region; and (4) for each partial imageregion, determining and coding respective coefficients of the new set oforthogonal basis functions, a sum of the orthogonal basis functionsmultiplied by the respective coefficients representing the gray scalevalues of the partial image region, the coded coefficients representingan approximation of the partial image region.
 5. A method as defined inclaim 4, wherein said step (1) includes subdividing the original imagesuch that the partial regions have similar structures and said set (2)includes the step of selecting the initial basis functions withreference to the local image content such that the local image contentis substantially describable with a small number of the coefficients ofthe orthogonal basis functions.
 6. A method as defined in claim 4,wherein said step (1) includes subdividing the original image such thatthe partial regions have similar structures and said step (2) includesthe step of selecting the set of initial basis functions from among aplurality of predetermined different sets of basis functions withreference to the local image content such that the local image contentis substantially describable with a small number of the coefficients ofthe orthogonal basis functions.
 7. A method as defined in claim 6,wherein at least one of the partial image regions contains softluminance transitions and the plurality of sets includes a set of basisfunctions which consists of polynomials, said step of selectingcomprising the step of selecting polynomials as the original basisfunctions for any partial image regions containing soft luminancetransitions.
 8. A method as defined in claim 6, wherein at least one ofthe partial image regions contains text and the plurality of setsincludes a set of basis functions which including of Walsh functions,said step of selecting comprising the step of selecting Walsh functionsas the set of original basis functions for any partial image regionscontaining text.
 9. A method as defined in claim 6 wherein at least oneof the partial image regions has a periodic texture and the plurality ofsets includes a set of cosine basis functions, said step of selectingcomprising the step of selecting cosine basis functions as the originalbasis functions for the at least one of the partial image regions havinga periodic texture.
 10. A method as defined in claim 4, furthercomprising the steps of producing a residual error image as a differencebetween a momentary image in video image sequence and an approximateimage generated from the image preceding it; and coding only suchpartial image regions of the residual error image by said steps (1)-(4)which are distinguished by high error amplitudes.
 11. A method as inclaim 4, wherein said step (1) includes subdividing the original imagesuch that the contours of the respective shapes are adapted to localimage structures.
 12. A method of transmitting coded image data,comprising the steps of:coding the image data according the steps ofclaim 6; coding contours of the partial image regions to obtain codedcontour information; and for each partial image region, transmitting thesmall number of coefficients and the coded contour information withoutthe orthogonal basis functions, the transmitted coefficients beingweighing factors of the subset of the set of orthogonal basis functionin an approximate representation of the partial image region.
 13. Amethod of transmitting coded image data, comprising the steps of:codingthe image data according the steps of claim 4; coding contours of thepartial image regions to obtain coded contour information; and for eachpartial image region, transmitting the coefficients of a subset of theset of orthogonal basis functions and the coded contour informationwithout transmitting the orthogonal basis functions, the transmittedcoefficients being weighing factors of the subset of the orthogonalbasis functions and providing an approximate representation of thepartial image region.
 14. A method of transmitting coded image data,comprising the steps of:coding the image data according the steps ofclaim 4; coding contours of the partial image regions to obtain codedcontour information; and for each partial image region, transmitting thecoefficients and the coded contour information without the orthogonalbasis functions, the coefficients being weighing factors of theorthogonal basis functions in a representation of the partial imageregion.
 15. A method as defined in claim 14, further comprising thesteps of receiving the transmitted coded coefficients and contourinformation, reconstructing the specific shape of the partial imageregion from the received contour information; regenerating the set oforthogonal basis functions from the specific shape; and reconstructingthe gray scale values within the partial image region as a weighted sumof the orthogonal basis functions, with the transmitted codedcoefficients constituting weighing factors of the orthogonal basisfunctions.